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by 

Howard  J.  Weiner 


I.  Introduction 

Three  stochastic  bang-bang  control  problems  are  considered ,  the 
predicted  miss,  the  linear  regulator,  and  a  simple  complete  observation 
model  ([1] ,[2] ,[3J),  which  have  been  solved  for  symmetric  constant  bound 
on  the  control  function  u(t)  ,  0  <  t  <  T.  Here  asymmetric,  finite 
bounds  on  u(t)  are  considered. 

To  state  these  problems  we  use  the  notation  and  definitions  of 
([1] » [2] , [3] ) . 

II.  Predicted  Miss 

Let  A(t),  B(t) ,  C(t)  be,  respectively,  dxd,  rxd,  and  dxd  matrix¬ 
valued  continuous  functions  on  I0,TJ  with 

(o,C(t)C'(t)o)  >  £<»,£>  >  0 
A  d 

for  all  t  €  [0,TJ  arid  «  6  R  ,  where  (a,b>  ■  2  a  b.  . 

i-1  1  1 


Denote  the  system  equation  by 

dX(t)  =>  A(t)X(t)dt  +  B(t)u(t)dt  +  C(t)dW(t) 


X(0)  -  G  €  Rd  (1) 

and  W(t)  is  standard  d-dimensional  Wiener  process.  Transpose  A  is  indicated 
by  A'. 

Assumption:  The  admissible  control  set  Q  consists  of  the  set 
of  processes 

u(t)  2  u(t,w)  such  that  -  »  <  a  <  f  (t)  <  u(t,w)  <  g(t)  <  b  <  ® 
where  f(t)  <  g(t)  are  bounded  continuous  non-random  given  functions  on 
[0,T]  %  and  f (0)  +  g(0)  -  0. 

The  Girsanov  Theorem  may  be  used  to  solve  (1)  in  law  (i.e.  there 
is  a  weak  solution.  The  boundedness  of  u(t,w)  insures  that  for  any 
given  u  €  G,  there  is  a  unique  solution  to  (1)  by  ([4]),  Theorem  1). 

Given  a  fixed  vector  Y,  the  cost  corresponding  to  u  6  G  is 

J(u)  -  Eu(Jt«Y,X(T)» 

where  for  each  u,  there  is  a  probability  space  (  with 

n  ■  C^[0,T],  X(t,w)  :  Cl  -*  R**  is  the  coordinate  map  X(t,w)  ■  w(t). 

Then  for  u  €  G,  on  (  Cl,?),  where  ?  -  a(X(s),  0  <  s  <  T)  is  such  that 
Pu[X(0)-G]  -  1  and  the  process  W(t,u)  defined  by 

W(t,u)  -J  c"l(s)dX(s)  -  J  C~1(s)[A(s)X(s)+B(s)U(s)}ds  is 
0  J0 

d-dimensional  Wiener  process.  Hence 

J(u)  ■  J^«Y,X(T)»  dPu  -  EuA«Y,X(T)». 

Jt  :  R  R+  has  these  properties : 


where 


(i)  &  is  even  :  £(x)  =■  i(-x). 


(ii)  i  is  continuously  differentiable  for  x  >  0 
and  &  ' (x)  >  0  all  x  >  0 

(iii)  t  (x)  »  0(exp  5(x|),  some  6  >  0. 

The  object  is  to  find  optimal  u^(t)  such  that 

J (u_)  *  min  J (u) . 

0  u€G 

Continuing  the  exposition  in  [2] ,  if  X(t)  «  x,  and  u(5)  =  0, 
t  <  5  <  T, 
then 

E[<Y,X(T)>jx<t)  -  x]  -  <v,  5'(t,X)x) 


The  cose  is  expressed  as 


J(u)  -  Eu[i(m(T))] 


Theorem  1  Under  conditions  above  in  II,  Che  optimal  control  Ug(t)  is 
expressible  by  components,  1  <  i  <  d,  with  =*  (u^,.  ..u^) , 

as 


<t) 


f  t(t)  +  g^t) 

2 


(g  <t)-f  (t»  ^  i  d  rT  n 

— - ^ -  sign'^[m(t)-i  2  j  bjt(s)(£x(s)+gjt(s))ds]bi(t)J  (3) 

X-l  t 


Proof.  First  assume  f^(t)  +  g^(t)  “  0,  all  1  <  i  <  d  and  set 


hi(t) 


8i (f )  “  ft(t) 
- 


In  this  case,  note  0  <  h^(t)  <  laj+jb|  ,  i  <  i  <  d  and  by  [4]  ,  since 
|u^(t)|<  h^(t),  then  (2)  has  a  unique  solution  with  u  ■*  u^. 

Set 

da(t)  ■  (b,u)dt  +  dv  then  it  follows  by  the  same  argument 

as  in  [2]  that 

u01(t)  -  -(^(t)  sign  (m(t)bi(t)^).  (4) 

is  optimal  for  the  symmetric  control  case.  In  general,  by  the  argument 
of  (12],  pp.  207-208),  one  may  invoke  symmetry  by  utilising  a 


4 


switching  curve  k(t)  such  that  if 

m(t)  *  m(t)  -  k(t),  then  it  would  follow  that 
<hn(t)  -  <b,u  -  — ^  )  dt  +  dv(t) . 

To  accomplish  (3). it  clearly  suffices  to  set 

*m\ 

and  k(T)  -  0, 

which  allows  J(m(T))  -  J(m(T)). 

Hence  (4)  -  (6)  suffice  for  the  proof  of  (3). 

III.  Linear  Regulator. 

A  one-dimensional  linear  regulator  problem,  following  [3]  is 
defined  as  follows:  The  one-dimensional  process  X(t,w)is 
given  by 

dX(t,w)  -  (aX(t,w)  +  u(t,w))dt  +  dW^t.w) 

with 

X(0.w)  -  X0(w) 

and  observation  equation 

dY(t,w)  -  cX(t,w)dt  +  <W2(t,w) 

and 

Y(0,w)  -  0 


for 


0  <  t  <  T 


a  >  0,  c  >  0  are  constants,  *re  In<*«P«Bdent 


where 


1 ' 


one-dimensional  Wiener  processes.  Let  (Cl.y.P^,)  be  Cl  =  C[0,TJ, 

5  -  a(X(s),  0  <  s  <  T),  and  P  correspond  to  W^jW^. 

Let  the  performance  index,  as  a  function  of  a  given  control  u  be 

T  T 

J(u)  -  J  E(X2(s))ds  s  \  X2(s,u)dP  (8) 

0  J0 

and  the  set  of  admissible  controls  0,  is  given  by 

a-{u|  |u(t,w)|  <  g(t),  0  <  t  <  T,  (9) 

with  continuous  g(t)  >0,  0  <  t  <  T. 

A  control  U_  €  CL  is  optimal  if 

J(uQ)  <  J(u)  all  u  6  Q. 

This  problem  may  be  recast  as  a  complete  observation  control  problem, 
following  ([3],  eq.  (3.19)  -  (3.21)).  The  new  state  variables  are 
R(t,w)  and  satisfy 

dR(t,w)  »  (aR(t,w)  +  u(t,w))dt 
+  c  p(t)  dW3(t,w) 

R(0,w)  -  E  XQ(w) 

T 

and  J(u)  -  J  E(R2(s))ds  (10) 

0 

where  W3  is  a  Wiener  process,  and  the  function  p(t)  satisfies  a 
Rlccati  equation 

^  -  2ap(t)  +  1  -c2p2(t)  0  <  t  <  T 

Qt 

p(0)  -  E(X02)  -  (EXq)2  ■  VarX0.  CU) 
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- . — jateafeBMfc- 


The  admissible  control  set  G  is  unchanged  and  it  is  noted  that  (11) 
does  not  depend  on  G  by  ({3],  eq.  (2.15)  -  (2.20)). 


Also  J(u)  -  j  E (R  (s))ds 
0 

Theorem  2  The  equation,  for  any  fixed  u  €  G, 


dV(t,v)  -  (aV(t,w)  +  u(t,w))dt 


+  cp(t)dW3(t,v) 


V(0,w)  -  EX0(w) 


has  a  unique  solution. 


Proof.  This  follows  from  the  boundedness  of  p,  u,  f,  g  in  [0,T]  by 
((4] ,  Theorem  1). 


Theorem  3  The  optimal  u  €  G  for  the  system  (10),  (11)  is  expressible 


v^(t,v)  »  -g(t)sign  X(t,w) , 


Proof.  This  follows  since  g(t)  >  0  factors  out  of  both  sides  of  ([3], 
eq.  (2.26)),  and  Theorem  2. 


IV  Complete  Observation 


Consider  a  one-dimensional  complete  observation  control  problem  with 
state  X(t,w),  control  u(t,w)  and  Wiener  process  W(t,w)  defined  by 


dX(t,w)  ■  u(t,w)dt  +  dW(t,w) 


and 


X(0,w)  =  x  . 

The  (CJ7P)  are  as  in  III. 

With  admissible  control  set 

0  *  Co!  Ju(t,w)|  <  g(t),  0  <  t  <  T} 

with  continuous  g(t)  >0,  0  <  t  <  T,  as  in  (9). 

The  performance  index  for  given  u  €  Q 
is 

J(u)  -  J  EjxU.v^dt  (15) 

0 

for  X  a  fixed  positive  integer.  The  object  is  to  find  uQ  €  a 
so  that 

J(tiQ)  <  J(u),  u  6  a. 

For  X  «  1,  2,  this  problem  was  solved  in  [lj. 

Theorem  4  Under  (9),  (14),  (15),  for  all  X  >  1,  the  solution  is 

u®(t,w)  -  -g(t)sign  X(t,w).  (16) 

Proof.  The  equation,  with  X(0,w)  -  x 

X(t,w)  -  -g(t)sign  X(t,w)dt  +  dW(t,w) 
has  a  unique  solution  ([4],  Theorem  1)  by  boundedness  of  g. 

The  reasoning  of  ([1],  pp.  93,  96,  eq.  (2.15))  implies  that  the  same 
optimal  Uq  holds  for  all  X  >  1  in  (15).  The  argument  is  as  in  Theorem  3. 
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